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Abstract. In this paper, we present the first snap-stabilizing message 
forwarding protocol that uses a number of buffers per node being inde- 
pendent of any global parameter, that is 4 buffers per link. The protocol 
works on a linear chain of nodes, that is possibly an overlay on a large- 
scale and dynamic system, e.g., Peer-to-Peer systems. Grids. . .Provided 
that the topology remains a linear chain and that nodes join and leave 
"neatly", the protocol tolerates topology changes. We expect that this 
protocol will be the base to get similar results on more general topologies. 
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1 Introduction 

These last few years have seen the development of large-scale distributed sys- 
tems. Peer-to-peer (P2P) architectures belong to this category. They usually 
offer computational services or storage facilities. Two of the most challenging 
issues in the development of such large-scale distributed systems are to come up 
with scalability and dynamicity. Scalability is achieved by designing protocols 
with performances growing sub-linearly with the number of nodes (or, proces- 
sors, participants). Dynamicity refers to distributed systems in which topological 
changes can occur, i.e., nodes may join or leave the system. 

Self- stabilization [1 is a general technique to design distributed systems that 
can tolerate arbitrary transient faults. Self-stabilization is also well-known to 
be suitable for dynamic systems. This is particularly relevant whenever the dis- 
tributed (self-stabilizing) protocol does not require any global parameters, like 
the number of nodes (n) or the diameter (D) of the network. With such a self- 
stabilizing protocol, it is not required to change global parameters in the program 
(n, etc) when nodes join or leave the system. Note that this property is also 
very desirable to achieve scalability. 

The end-to-end communication problem consists in delivery in finite time 
across the network of a sequence of data items generated at a node called the 
sender, to a designated node called the receiver. This problem is generally split 
into the two following problems: {i) the roi^^m^ problem, i.e., the determination 
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of the path fohowed by the messages to reach their destinations; {ii) the message 
forwarding problem that consists in the management of network resources in 
order to forward messages. The former problem is strongly related to the problem 
of spanning tree construction. Numerous self-stabilizing solutions exist for this 
problem, e.g., |2l3l4] . 

In this paper, we concentrate on the latter problem, i.e., the message for- 
warding problem. More precisely, it consists in the design of a protocol managing 
the mechanism allowing the message to move from a node to another on the path 
from the sender A to the receiver B. To enable such a mechanism, each node 
on the path from A to B has a reserved memory space called buffers. With a 
finite number of buffers, the message forwarding problem consists in avoiding 
deadlocks and livelocks (even assuming correct routing table). Self-stabilizing 
solutions for the message forwarding problem are proposed in [5"6' . Our goal is 
to provide a snap-stabilizing solution for this problem. A snap-stabilizing pro- 
tocol [7 guarantees that, starting from any configuration, it always behaves ac- 
cording to its specification, i.e., it is a self-stabilizing algorithm which is opti- 
mal in terms of stabilization time since it stabilizes in steps. Considering the 
message-forwarding problem, combined with a self-stabilizing routing protocol, 
snap-stabilization brings the desirable property that every message sent by the 
sender is delivered in finite time to the receiver. By contrast, any self-stabilizing 
(but not snap-stabilizing) solution for this problem ensures the same property, 
"eventually" . 

The problem of minimizing the number of required buffers on each node is 
a crucial issue for both dynamicity and scalability. The first snap-stabilizing 
solution for this problem can be found in [8 . Using n buffers per node, this 
solution is not suitable for large-scale system. The number of buffers is reduced to 
D in [9 , which improves the scalability aspect. However, it works by reserving the 
entire sequence of buffers leading from the sender to the receiver. Furthermore, 
to tolerate dynamicity, each time a topology change occurs in the system, both 
of them would have to rebuild required data structures, maybe on the cost of 
loosing the snap-stabilisation property. 

In this paper, we present a snap-stabilizing message forwarding protocol that 
uses a number of buffers per node being independent of any global parameter, 
that is 4 buffers per link. The protocol works on a linear chain of nodes, that 
is possibly an overlay on a large-scale and dynamic system e.g., Peer-to-Peer 
systems. Grids. . . Provided that (i) the topology remains a linear chain and {ii) 
that nodes join and leave "neatly", the protocol tolerates topology changes. 
By ^^neatl2/\ we mean that when a node leaves the system, it makes sure that 
the messages it has to send are transmitted, i.e., all its buffers are free. We 
expect that this protocol will be the base to get similar results on more general 
topologies. 

The paper is structured as follow: In Section [21 we define our model and some 
useful terms that are used afterwards. In Section [3l we first give an informal 
overview of our algorithm, followed by its formal description. In Section HI we 
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prove the correctness of our algorithm. Dynamicity is discussed in Section [5l We 
conclude the paper in Section [6l 

2 Model and definitions 

Network. We consider a network as an undirected connected graph G = (F, E) 
where V is the set of nodes (processors) and E is the set of bidirectional com- 
munication links. A link (p, q) exists if and only if the two processors p and q 
are neighbours. Note that, every processor is able to distinguish all its links. To 
simplify the presentation we refer to the link (p, q) by the label q in the code of 
p. In our case we consider that the network is a chain of n processors. 



Computational model. We consider in our work the classical local shared 
memory model introduced by Dijkstra fTO^ known as the state model. In this 
model communications between neighbours are modelled by direct reading of 
variables instead of exchange of messages. The program of every processor con- 
sists in a set of shared variables (henceforth referred to as variable) and a finite 
number of actions. Each processor can write in its own variables and read its 
own variables and those of its neighbours. Each action is constituted as follow: 

< Label >::< Guard > ^ < Statement > 

The guard of an action is a boolean expression involving the variables of 
p and its neighbours. The statement is an action which updates one or more 
variables of p. Note that an action can be executed only if its guard is true. Each 
execution is decomposed into steps. 

The state of a processor is defined by the value of its variables. The state of 
a system is the product of the states of all processors. The local state refers to 
the state of a processor and the global state to the state of the system. 

Let y ^ C and A an action of p {p e V). A is enabled for p in y if and 
only if the guard of A is satisfied by p in y. Processor p is enabled in y if 
and only if at least one action is enabled at p in Let P be a distributed 
protocol which is a collection of binary transition relations denoted by on 
C. An execution of a protocol P is a maximal sequence of configurations e = 
y^yi-'-yiyi+i • • • such that, V z > 0, ^ t/^+i (called a step) if exists, 
else yi is a terminal configuration. Maximality means that the sequence is either 
finite (and no action of P is enabled in the terminal configuration) or infinite. 
All executions considered here are assumed to be maximal. ^ is the set of all 
executions of P. Each step consists on two sequential phases atomically executed: 
(i) Every processor evaluates its guard; {ii) One or more enabled processors 
execute its enabled actions. When the two phases are done, the next step begins. 
This execution model is known as the distributed daemon [11]. We assume that 
the daemon is weakly fair^ meaning that if a processor p is continuously enabled^ 
then p will be eventually chosen by the daemon to execute an action. 
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In this paper, we use a composition of protocols. We assume that the above 
statement (n) is apphcable to every protocol. In other words, each time an 
enabled processor p is selected by the daemon, p executes the enabled actions of 
every protocol. 

Snap- Stabilization. Let be a task, and Sr a specification of F . A protocol 
P is snap-stabilizing for Sr if and only if \/r G ^, satisfies Sp- 

Message Forwarding Problem. The message forwarding problem is specified 
as follows: 

Specification 1 (SP) A protocol P satisfies SP if and only if the following 
two requirements are satisfied in every execution of P: 

1. Any message can he generated in a finite time; 

2. Any valid message is delivered to its destination once and only once in a 
finite time. 

Buffer Graph In order to conceive our snap stabilizing algorithm we will use 
a structure called Buffer Graph introduced in [12]. A Buffer Graph is defined 
as a directed graph where nodes are a subset of the buffers of the network and 
links are arcs connecting some pairs of buffers, indicating permitted message 
fiow from one buffer to another one. Arcs are permitted only between buffers 
in the same node, or between buffers in distinct nodes which are connected by 
communication link. 

Let us define our buffer graph (refer to Figure [1]): 

C2 



OUTp (q) INp(q') 




UTl 



INp(q) OUTp(q') 



CI 

Fig. 1. Buffer Graph 

Each processor p has four buffers, two for each link (p, q) such diS q e Np (except 
for the processors that are at the extremity of the chain that have only two 
buffers, since they have only one link). Each processor has two input buffers de- 
noted by INp{q)^ INp{q') and two output buffers denoted by OUTp{q)^ OUTp{q') 
such as (7, q' G Np and q ^ q' (one for each neighbour). The generation of a mes- 
sage is always done in the output buffer of the link (p, q) so that, according to 
the routing tables, q is the next processor for the message in order to reach the 
destination. Let us refer to n6(m, b) as the next buffer of Message m stored in 
b,b e {INp{q) V OUTp{q)}, q e Np. We have the following properties: 
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1. nb{mJNp{q))=OUTq{p) 

2. nb{m,OUTp{q) = INq{p) 

3 Message Forwarding 

In this section, we first give the idea of our snap stabihzing message forwarding 
algorithm in the informal overview, then we give the formal description followed 
by the correctness proofs. 

3.1 Overview of the algorithm 

In this section, we provide an informal description of our snap stabilizing message 
forwarding algorithm that tolerates the corruption of the routing tables in the 
initial configuration. 

To ease the reading of the section, we assume that there is no message in 
the system whose the destination is not in the system. This restriction is not a 
problem as we will see in Section O 

We assume that there is a self-stabilizing algorithm, Rtables^ that calculates 
the routing tables and runs simultaneously to our algorithm. We assume that 
our algorithm has access to the routing tables via the function Nextp{d) which 
returns the identity of the neighbour to which p must forward the message to 
reach the destination d. To reach our purpose we define a buffer graph on the 
chain which consists of two chains, one in each direction (CI and C2 refer to 
Figure [1]) . 

The overall idea of the algorithm is as follows: When a processor wants to 
generate a message, it consults the routing tables to determine the next neigh- 
bour by which the message will transit in order to reach the destination. Note 
that the generation is always done in the Output buffers. Once the message is 
on the chain, it follows the buffer chain (according to the direction of the buffer 
graph) and if the messages can progress enough in the system (move) then it 
will either meet its destination and hence it will be consumed in a finite time or 
it will reach the input buffer of one of the processors that are at the extremity 
of the chain. In the latter case, if the processor that is at the extremity of the 
chain is not the destination then, that means that the message was in the wrong 
direction. The idea is to change the direction of the message by copying it in the 
output buffer of the same processor (directly (UTl) or using the extra buffer 
(UT2), refer to Figure [J). 

Note that if the routing tables are stabilized and if all the messages are in 
the right direction then all the messages can move on CI or C2 only and no 
deadlock happens. However, in the opposite case (the routing tables are not 
stabilized or some messages are in the wrong direction), deadlocks may happen 
if no control is introduced. For instance, suppose that in the initial configuration 
all the buffers, uncluding the extra buffer of /7T2, contain different messages 
such that no message can be consumed. It is clear that in this case no message 
can move and the system is deadlocked. Thus in order to solve this problem we 
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have to delete at least one message. However, since we want a snap stabilizing 
solution we cannot delete a message that has been generated. Thus we have to 
introduce some control mechanisms in order to avoid this situation to appear 
dynamically (after the first configuration). In our case we decided to use the 
PIF algorithm that comprises two main phases: Broadcast (Flooding phase) and 
Feedback (acknowledgement phase) to control and avoid deadlock situations. 

Before we explain how the PIF algorithm is used, let us focus on the mes- 
sage progression again. A buffer is said to be free if and only if it is empty (it 
contains no message) or contains the same message as the input buffer before it 
in the buffer graph buffer. For instance, if INp{q) = OUTq{p) then OUTq{p) is 
a free buffer. In the opposite case, a buffer is said to busy. The transmission of 
messages produces the filling and the cleaning of each buffer, i.e., each buffer is 
alternatively free and busy. This mechanism clearly induces that free slots move 
into the buffer graph, a free slot correspondingto a free buffer at a given instant. 
The moving of free slots is shown in Figure QEI- Notice that the free slots move 
in the opposite direction of the message progression. This is the key feature on 
which the PIF control is based. 





(a) The input buffer of p is free. Node 
p can copy the message a. 



Free Buffer 



(h) The output buffer of p' is free. 
Node p can copy the message h. 




(c) The input buffer of p is free. Node 
p' can copy the message c. 



(d) The output buffer of q is free. Node 
q can copy the message d. 



Fig. 2. An example showing the free slot moving. 



When there is a message that is in the wrong direction in the Input buffer 
of the processor po, Po copies this message in its extra buffer releasing its Input 
buffer and it initiates a PIF wave at the same time. The aim of the PIF waves is 
to escort the free slot that is in the input buffer of po in order to bring it in the 
Output buffer of po- Hence the message in the extra buffer can be copied in the 
output buffer to become in the right direction. Once the PIF wave is initiated no 
message can be generated on this free slot, at each time the Broadcast progresses 
on the chain the free slot moves as well following the PIF wave (the free slot 
moves by transmitting messages on CI (refer to Figured]). In the worst case, the 

^ Note that in the algorithm, the actions {b) and (c) are executed in the same step 
(refer to the guarded action i?3). 
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free slot is the only one, hence by moving the output buffer of the other extremity 
of the chain p becomes free. Depending on the destination of the message that is 
in the input buffer of either this message is consumed or copied in the Output 
buffer of p. In both cases the input buffer of p contains a free slot. 

In the same manner during the feedback phase, the free slot that is in the 
input buffer of the extremity p will progress at the same time as the feedback 
of the PIF wave. Note that this time the free slot moves on C2 (see Figure [T]). 
Hence at the end of the PIF wave the output buffer that comes just after the 
extra buffer contains a free slot. Thus the message that is in the extra buffer can 
be copied in this buffer and deleted from the extra buffer. Note that since the 
aim of the PIF wave is to bring the free slot in the output buffer of po then when 
the PIF wave meets a processor that has a free buffer on C2 the PIF wave stops 
escorting the previous free slot and starts the feedback phase with this second 
free slot (it escorts the new free slot on C2). Thus it is not necessary to reach 
the other extremity of the chain. 

Now, in the case where there is a message in the extra buffer of po such as no 
PIF wave is executed then we are sure that this message is an invalid message 
and can be deleted. In the same manner if there is a PIF wave that is executed 
such that at the end of the PIF wave the Output buffer of po is not free then 
like in the previous case we are sure that the message that is in the extra buffer 
is invalid and thus can be deleted. Thus when all the buffers are full such as all 
the messages are different and cannot be consumed, then the extra buffer of po 
will be released. 

Note that in the description of our algorithm, we assumed the presence of 
a special processor po. This processor has an Extra buffer used to change the 
direction of messages that are in the input buffer of po however their destination 
is different frompo- In addition it has the ability to initiate a PIF wave. Note also 
that the other processors of the chain do not know where this special processor 
is. A symmetric solution can also be used (the two processors that are at the 
extremity of the chain execute the same algorithm) and hence both have an 
extra buffer and can initiate a PIF wave. The two PIF wave initiated at each 
extremity of the chain use different variable and are totally independent. 

3.2 Formal description of the algorithm 

We first define in this section the different data and variables that are used in 
our algorithm. Next, we present the PIF algorithm and give a formal description 
of the linear snap stabilizing message forwarding algorithm. 

Character ^ ? Mn the predicates and the algorithms means any value. 

- Data 

• n is a natural integer equal to the number of processors of the chain. 

• / = {0, n — 1} is the set of processors' identities of the chain. 

• Np is the set of identities of the neighbours of the processor p. 

— Message 
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• (m, d^c): m contains the message by itself, i.e., the data carried from the 
sender to the recipient, d e I is the identity of the message recipient, and 
c is a color number given to the message to avoid duplicated deliveries. 

— Variable 

• In the forwarding algorithm 

* INp{q): The input buffer of p associated to the link (p, q). 

* OUTp{q): The output buffer of p associated to the link (p, q). 

* EXTpi The Extra buffer of processor p which is at the extremity of 
the chain. 

• In the PIF algorithm 

* Sp = {B V F V C^q) refers to the state of processor p^ q is a, pointer 
to a neighbour of p. 

— Input/Output 

• Requestpi Boolean, allows the communication with the higher layer, it 
is set at true by the application and false by the forwarding protocol. 

• PIF-Requestp'. Boolean, allows the communication between the PIF and 
the forwarding algorithm, it is set at true by the forwarding algorithm 
and false by the PIF algorithm. 

• The variables of the PIF algorithm are the input of the forwarding algo- 
rithm. 

— Procedure 

• Nextp {d) : refers to the neighbour of p given by the routing table for the 
destination d. 

• Deliver p{m): delivers the message m to the higher layer of p. 

• Choice{c): chooses a color for the message m which is different from the 
color of the message that are in the buffers connected to the one that 
will contain m. 

— Predicate 

• Consumptionp{q^ m): INp{q) = (m, d^c) A d = p A OUTq{p) ^ (m, c) 

• leafp{q): Sq = {B, ?) A (V G Np/{q}, Sq> ^ {B,p) A (con sumption p{q) 
V OUTp{q') = e V OUTp{q^) = /iV,,(p))). 

• NO-PIFp: Sp = (C, NULL) AWqe Np, Sq 7^ (5, ?). 

• init-PIF: Sp = (C.NULL) A (Vg G Np, Sq = {C.NULL)) A 
PIF-Requestp = true. 

• Inter -transp{q): INp{q) = {m,d,c) A d ^ p A OUTq{p) ^ INp{q) A 
{3q' e Np/{q}, OUTp{q') = e V OUTp{q^) = INq>{p)). 

• internal p{q)'. p ^ po A ^ leafp{q). 

• Road-Changep{m): p = po A INp{q) = (m, c) A d^ p A EXTp = e A 
OUT,{p) ^ IN^iq). 

• V T Action G C, 5, we define T Action-initiator p the predicate: p = po A 
(the garde of TAction in p is enabled). 

• V Tproc G {internal, leaf} and TAction G T-Action-TproCp{q) 
is defined by the predicate: TproCp{q) is true A TAction of p is enabled. 

• PIF-SynchrOp{q): {Bq-internalp V Fq-leafp V Fq-internalp) A Sq = 
{B,7). 
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— We define a fair pointer that chooses the actions that will be performed on 
the output buffer of a processor p. (Generation of a message or an internal 
transmission). 



Algorithm 1 PIF 

— For the initiator (po) 

• B-Action:: init-PIF Sp := (B, — 1), PIF-Requestp := false. 

• C-Action:: Sp = {B, -1) A Vg G Np, Sq = (F, ?) Sp := (C, NULL). 

— For the leaf processors: leafp{q) = true V \Np\ = 1 

• F-Action:: Sp = {C,NULL) Sp := (F, g). 

• C-Action:: Sp = (F, ?) A Vg G Np, 5^ = (F V C, ?) Sp := (C, NULL). 

— For the processors 

• B-Action:: 3\q G Np, Sq = (B, ?) A = (C, ?) A Vg' G A^p/{g}, S^, = (C, ?) 

• F-Action:: 5p = (B, g) A 5g = (B, ?) A Vq' G A/'p/{g}, 5^/ = (F, ?) ^ 5p (F, g). 

• C-Action:: 5^ = (F, ?) A Vg' G A/'p, 5^/ = (F V C, ?) ^ := (C, NULL). 

— Correction (For any processor) 

• Sp ^ {B, q) A Sq ^ {FV C, ?) Sp := (C, NULL). 

• ^ea/p((?) A 5p = (B, g) ^ 5p := (F, g). 



4 Proof of Correctness 

In this section, we prove the correctness of our algorithm. We first show that 
starting from an arbitrary configuration, our protocol is deadlock free. Next, we 
show that no node can be starved of generating a new message. Next, we show 
the snap-stabilizing property of our solution by showing that, starting from any 
arbitrary configuration and even if the routing tables are not stabilized, every 
valid message is delivered to its destination once and only once in a finite time. 
Let us first state the following lemma: 

Lemma 1 The PIF protocol (Algorithm 1) is snap- stabilizing. 

Proof. Note that the PIF algorithm introduced here is similar to the one 
proposed in [7] which is a snap stabilizing algorithm. The new thing is that we 
introduced the idea of dynamic leafs, processors that satisfy some properties and 
act like a physical leaf (they execute the F-action once they have a neighbor in a 
broadcast phase). Hence instead of reaching all the nodes of the chain, the PIF 
wave stops advancing when it meets a dynamic leaf. Note that once an internal 
processor p executes the B-Action, it cannot execute the F-Action unless is has 
a neighbor q such as Sq = {F^p) (it cannot become a leaf) since to execute the 
F-action by any processor p, Sp = (C, NU LL) or for the internal processor that 
executes the B-Action Sp = {B^q) (q G A/p, Sq = {B^p')). Thus no processor 
becomes a dynamic leaf of the PIF wave once it executed the B-Action of the 
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Algorithm 2 Message Forwarding 

— Message generation (For every processor) 

Rl:: Requestp A Nextp{d) = q A [OUTp{q) = e V OUTp{q) = INq{p)] A NO-PI Fp 
OUTp{q) := {m, d, choice (c)) , Requestp := false. 

— Message consumption (For every processor) 

R2:: 3q G Np, 3m G M; C on sumption p{q^ m) deliver p{m)^ INp{q) := OUTq{p). 

— Internal transmission (For processors having 2 neighbors) 

R3:: 3q G Np, 3m e M , 3d e /; Inter-transp{q, m, d) A {NO-PIFp V PI F-SynchrOp{q)) 
OUTp{q') := {m, d, choice {c)) , INp{q) := OUTq{p). 

— Message transmission from g to p (For processors having 2 neighbors) 

R4:: INp{q) = e A OUTq{p) 7^ e A {NO-PIFp V PI F-SynchrOp{q)) INp{q) := OUTq{p). 

— Erasing a message after its transmission (For processors having 2 neighbors) 

R5:: 3q G Np, OUTp{q) = INq{p) A (Vg' G Np \ {g}, INp{q') = e) A 
{NO-PIFp V PIF-SynchrOp{q)) OUTp{q) := e, INp{q') := OUTq, {p). 

— Erasing a message after its transmission (For the extremities) 

R5':: Np = {q} A OUTp{q) = INq{p) A INp{q) = e A {{p = po) ^ {EXTp = e)) A 
{NO-PIFp V PIF-SynchrOp{q)) OUTp{q) := e, INp{q) := OUTq{p). 

— Road change (For the extremities) 

• R6:: Road-Changep{m) A [OUTp{q) = e V OUTp{q) = INq{p)] OUTp{q) : = 
{m,d,choice{c)), INp{q) := OUTq{p). 

• R7:: Road-Changep{m) A OUTp{q) ^ e A PIF-Requestp = false PIF-Requestp : = 
true. 

• R8:: Road-Changep{m) A OUTp{q) 7^ e A PIF-Requestp A B-initiator EXTp : = 
INp{q), INp{q) := OUTq{p). 

• R9:: p = po A EXTp 7^ e A [OUTp{q) = e V OUTp{q) = INq{p)] A C-Initiator 
OUTp{q) := EXTp, EXTp := e. 

• RIO:: p = Po A EXTp / e A OUTp{q) ^ e A OUTp{q) 7^ INq{p) A C-Initiator 
EXTp := e. 

• Rll:: \Np\ = 1 A p 7^ A INp{q) = {m, d, c) A d p A OUTp{q) = e A OUTq{p) ^ INp{q) 

OUTp{q) := {m, d, choice{c)) , INp{q) := OUTq{p). 

— Correction (For po) 

• R12:: p = po A EXTp ^ e A Sp ^ {B, -1) EXTp = e. 

• R13:: p = pQ A Sp = {B, ?) A PIF-Request = true PIF-Request = false. 

• R14:: p = po A Sp = (C, ?) A PIF-Request = true A [{INp{q) = {m,d,c) A d = p) V 
INp{q) = e] ^ PIF-Request = false. 



same PIF wave. In another hand, note that the variable PIF-Request is a shared 
variable between the PIF algorithm and the forwarding algorithm, its role is to 
give the signal to the initiator to initiate the PIF wave. Hence we can deduct by 
analogy that the PIF algorithm proposed here is a snap stabilizing algorithm. □ 

We now show (Lemma [2]) that the extra buffer located at po cannot be 
infinitely continuously busy. As explained in Section [H this solves the problem 
of deadlocks. 

Lemma 2 // the extra buffer of the processor po (EXTp^) which is at the ex- 
tremity of the chain contains a message then this buffer becomes free after a 
finite time. 

Proof. We know from Lemma [1] that each time po launches a PIF wave, 
then this wave terminates. When this happens, there are two cases: 
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1. Sp = {C, NULL). In this case R12 is enabled on p. Since the daemon is 
weakly fair we are sure that R12 will be executed in a finite time. Thus 
EXTp will be free in a finite time too. 

2. Sp = (5,?). In this case^ a PIF wave is executed on the chain. Note that 
p is the initiator (it is eventually considered as the initiator). According to 
Lemma[T]the PIF is a Snap stabilizing algorithm. Hence, there will be a time 
when Sq = (F^p)^ q G Np. Two sub cases are possible: 

- Either OUTp{q) = e or OUTp{q) = INq{p). In this case R9 is enabled 
on p. Since the daemon is weakly fair, this rule will be executed in a 
finite time. Hence the message that is in the extra buffer will be copied 
in OUTp{q) and deleted from EXTp (see Rule R9). Hence EXTp = e. 

- OUTp{q) 7^ e and OUTp{q) INq{p). Since RIO is enabled on p and the 
daemon is weakly fair, RIO will be executed in a finite time. Thus, the 
message that is in the extra buffer is deleted. 

From the cases above, we deduct that in the case where the extra buffer 
of p contains a message, then this message will be either deleted or copied in 
OUTp{q). Hence EXTp will be free in a finite time and the lemma holds. □ 

We deduce from Lemma [2] that if the routing tables are not stabilized and if 
there is a message locking the Input buffer of po, then this message is eventually 
copied in the extra buffer. Since the latter is infinitely often empty (Lemma [2] 
again) . 

From now on, we suppose that no generated message is deleted (we prove 
this property latter). 

Lemma 3 All the messages progress in the system even if the routing tables are 
not stabilized. 

Proof. Note that if 3q G Np, INp{q) is free then if there is a message 
in OUTq{p), then this message is automatically copied in INp{q). Hence it is 
sufficient to prove that the input buffer are free in a finite time. Thus Let's 
prove that Vp G /, when there is a message in INp{q), this message is deleted 
from INp{q) in a finite time {q G Np). 

Note that the input-buffers are all at an even distance from the input buffer 
of the processor po. Let define 6 as the distance between the input buffer of the 
processor p and the input buffer of processor po (In the direction of the buffer 
graph taken in account UTl). The lemma is proved by induction on S. We define 
for this purpose the following predicate Ps: 

If there is a message mh in INp{q) such as INp{q) is at distance S from the 
input buffer of po then one of these two cases happens: 

— m is consumed and hence delivered to its destination. 

— m is deleted from the input buffer and copied either in EXTp or OUTp{q') 
in a finite time. 
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Initialization. Let's prove the result for Pq. Suppose that there is a message 
m in INp{q) such as p = po and q e Np. Two cases are possible according to the 
destination (d) of m: 

— d = p. In this case, since the daemon is weakly fair and since R2 keep being 
enabled on p then R2 will be executed on p in a finite time and the message 
m in INp{q) is consumed. Thus Pq is true. 

— d ^ p. Since the daemon is weakly fair we are sure that p will be activated. 
Two cases are possible: 

• OUTp{q) = e or OUTp{q) = INq{p). In this case R6 is enabled on p. 
Hence the message m is copied in OUTp{q) and deleted from INp{q) 
since a new value overwrite it (see R6). Thus Pq is true. 

• OUTp{q) ^ e and OUTp{q) ^ INq{p). According to Lemma El EXTp 
will be free in a finite time. In another hand since the PIF is a snap 
stabilizing algorithm (refer to Lemma [TJ we are sure that the B- Action 
of the initiator will be enabled on p in a finite time). Hence the message 
m will be copied in this case in EXTp and deleted from INp{q) (Note 
that in the case where PIF-Reaquest = false then it is set at true (see 
rule R7)). Thus Pq is true. 

In both cases either the message is consumed or it is removed from INp{q). 
Thus Pq is true. 

Induction, let (5 > 1. We assume that P25 is true and we prove that P25-\-2 is 
true as well (Recall that the input buffers are at an even distance from the input 
buffer ofpo). Let INq{p) be the buffer at distance 25 from the input buffer of po 
and INp{q') the one that is at distance 2(5 + 2 and contains the message m' . 
In the case where the destination of m' is p then it will be consumed in a 
finite time (the daemon is weakly fair and R2 keep being enabled on p. Thus 
p will execute R2 in a finite time). Hence P25+2 is true. In the other case (the 
destination of m' is different from p), since P25 is true then if there is a message 
m in INq{p) then we are sure that this message will be either consumed or copied 
in OUTq{q"). Thus INq{p) = OUTp{q) {OUTp{q) is free). Two cases are possible 
according to the rule that is executed on OUTp{q) (depending on the value of 
the pointer on OUTp{q)) : 

1. p executes R3. In this case the message m' is copied in OUTp{q) and deleted 
from INp{q') since a new value overwrite it (refer to Rule i?3). Hence P25+2 
is true. 

2. p executes R2 (it generates a message). Hence OUTp{q) = m" {m" is the 
message generated by p). However, since P25 is true, then the message in 
INq{p) will be deleted from the buffer {q performs either an internal trans- 
mission or consume the message). Hence INq{p) = OUTp{q) in a finite time. 
Nevertheless, since p generated a message in the previous step, we are sure 
that it will execute R3 (since the pointer on the output buffer OUTp{q) is 
fair). Thus we retrieve case[TJ 
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From the proof above, we deduct that ah the messages in the chain progress 
in the system and no deadlock happens even if the routing tables are corrupted. 

□ 

Let us call a valid PIF wave every PIF wave that is initiated by the processor 
po at the same time as executing R8. 

Lemma 4 For every valid PIF wave, when the C -Action is executed in the 
initiator either OUTp{q) = INq{p) or OUTp{q) = e. 

Proof. The idea of the proof is as follows: 

— We prove first that during the broadcast phase there is a synchrony between 
the PIF and the forwarding algorithm. Note that when the message that was 
in the input buffer of the initiator is copied in the extra buffer, the input 
buffer becomes free. The free slot in that buffer progresses in the chain at 
the same time as the broadcast of the PIF wave. 

— Once the PIF reaches a leaf, a new buffer becomes free in C2 (refer to Figure 

m- 

— As in the broadcast phase, there is a synchrony between the PIF and the 
forwarding algorithm during the feedback phase. (The feedback will escort 
the new free slot on C2 to the output buffer of po-) 

Let's prove that during the broadcast phase there is a synchrony between the 
PIF and the forwarding algorithms. Let's define for this purpose A the distance 
between the processor p and the processor po. We're going to prove the result 
by induction on A. let's define fot this purpose the predicate Fx as follow: 

When the PIF wave is initiated and for each extra processor p {^leafAp ^ po) 
that executes the B-action we have: 3! g G TVp, Sq = (5,?), Sp = {B^q) A 
INp{q') = OUTq^p) A Sq> = (C.NULL) {q' e Np/{q}). 

Initialization. Let's prove that Pi is true. Since the PIF wave is valid, 
when the PIF wave was initiated, RS was executed at the same time. Hence, 
the message that was in INp^i^q) was copied in EXTp^^ INp^i^q) = OUTq{po)^ 
SpQ = (5, —1) and Sq = {C^ NULL). Since q is not a leaf only R3 or R5 and 
possibly R2 are enabled on q. Note that after the execution of one of these rules 
INq{q') = OUTq>{q) {q' G Nq/ {po}). In another hand and Rb are not enabled 
only if the B-action of the internal processor is enabled as well. Thus when the 
B- Action is executed (we are sure that this will happen since the PIF algorithm 
is snap stabilizing and the daemon is weakly fair) either i?3 or Rb (possibly 
R2) are executed at the same time (Recall that when two actions from the PIF 
and the forwarding algorithm are enabled on the same processor at the same 
time they are both executed). Hence INq{q') = OUTq'{q), Sq = (B^po) and 
Sq' = (C, NULL). Thus Pi is true. Note that if R2 is executed alone before the 
B- Action then either R3 or R5 are still enabled on q. Hence when the B- Action 
is executed one of these two actions are executed as well. 
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Induction. Let A > 2. We assume that Pa is true and we prove that Px+i 
is true as weh. Let q and p be the processors that are at distance A and A + 1 
respectively from the processor po. Since Px is true, when the B- Action of the 
internal processor is executed on Sq = {B^q")^ INq{p) = OUTp{q) and Sp = 
{C^ NULL). In the same manner as the proof of Pi, R3 or R5 and possibly R2 
is enabled on p. Note that R3 or R5 keep being enabled unless they are executed 
(Note that Rl cannot be executed since there is a PIF wave that is executed 
in the neighbourhood. Thus no message is generated). In another hand R3 and 
R5 cannot be executed unless the B- Action is enabled as well. Hence when the 
B- Action is executed either R3 or R5 or R2 is executed at the same time. Hence 
INp{q') = OUTq^p), Sp = (P,g) and Sq' = (C.NULL). Thus Pa+i is true. 

We can deduct that for the last processor p that is an internal processor 
{p 7^ Po and - leaf), Sq = {B,q") and INp{q') = OUTq\p), Sp = {B,q) and 
Sq' = {C^ NULL). Since p is the last processor which is an internal processor 
then q^ is a leaf. Two cases are possible: 

— The leaf is the processor q' that is at the extremity of the chain such as 
q' ^ Pq. Either R2 or Rll are enabled on q' . Note that the F- Action is 
enabled as well since q^ is a leaf and Sp — (P,?). When one of these two 
rules is executed with the F- Action, Sqi = (P,p) and INq\p) = OUTp{q'). 

— The leaf q' is not the processor that is at the border of the chain. In this 
case either R2 or R3 or RA are enabled. Recall that the F-action is enabled 
as well. Then once the F- Action of the internal processor and one of these 
rules are executed, Sq' = (P,p) and INq'{p) = OUTp{q'). 

Note that in both cases, once the leaf q' executed the F- Action we have the 
following property: Sq' — (F^p) and INq'{p) = OUTp{q'). Now in the same 
manner that we proved the synchrony between the PIF and the forwarding 
algorithm during the broadcast phase. We prove the synchrony between these two 
algorithms during the feedback phase. The proof is by induction on A using the 
following predicate: For every internal processor p that executes the F- Action, 
Sp = F,q, Sq' = {F,p), Sq' = (P,?) and INp{q) = OUTq{p) {q,q' eNp.q^ q'). 
Then when the last internal processor executes that F-action (note that the 
last processor is the one that is neighbour to the initiator) these properties are 
verified. Hence the Output buffer of the initiator is becomes free and the lemma 
holds. 

□ 

In the remainder, we say that a message is in a suitable buffer if the buffer 
is on the right direction to its destination. A message is said to be deleted if it 
is removed from the system without being delivered. 

Let m be a message. According to Lemma [3l m progresses in the system 
(no deadlock happens and no message stays in the same buffer indefinitely). So, 
if m is in a buffer that is not suitable for it, then m progresses in the system 
according to the buffer graph. Thus, it eventually reaches an extremity, which 
changes its direction. Now, m is ensured to reach its destination, leading to the 
following lemma: 
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Lemma 5 For every message that is not in a suitable buffer, it will undergo 
exactly a single route change. 

Proof. Let m be a message. According to Lemma [3l m progresses in the 
system (no deadlock happens and no message stays in the same buffer indefi- 
nitely). So, if m is not in a buffer that is not suitable for it, then m progresses 
in the system according to the buffer graph. Two cases are possible: 

— m will be in a finite time in the Input buffer of the processor po. Since the 
message is in a buffer that is not suitable for it, po is not the destination 
of m. However we are sure that EXTp will be free in a finite time (refer 
to Lemma [2]) and that the B- Action will be enabled on po in a finite time 
too (The PIF is a snap stabilizing algorithm). Hence the message in INp^i^q) 
will be copied in EXTp^ and a PIF wave is initiated at the same time. In 
another hand, it has been shown in Lemma 3] that in the case of a valid 
PIF wave (Note that this is our case) when the PIF ends {Sq = (F,po)) and 
C-Initiator is enabled on po (we are sure that this will happen since the PIF 
is snap stabilizing) and OUTp{q) becomes free. Hence R9 is enabled on po 
and the message that is in the extra buffer can be put in the output buffer 
of Po and deleted from the extra buffer. Note that since the network is a 
chain and po is at the extremity of this chain, we are sure that the message 
will meet its destination since it can visit all the processors. Hence no other 
changes route are done. 

— The message will reach the input buffer of the processor p' that is at the 
other extremity of the chain {p' ^ 0). Since the messages progress in the 
system (see Lemma [3]) OUTp>{q') will be free in a finite time. Hence when 
a message that is not intended to q is in INq{q') and since the daemon is 
weakly fair, we are sure that q will execute i?ll in a finite time. Thus the 
message will be copied in OUTq{q') and deleted from INq{q'). Now since q 
is at the extremity of the chain, the message will meet its destination hence 
no other route change is performed and the lemma holds. 

□ 

Once the routing tables are stabilized, every new message is generated in a 
suitable buffer. So, it is clear from Lemma [5] that the number of messages that 
are not in a suitable buffer strictly decreases. The next lemma follows: 

Lemma 6 When the routing tables are stabilized and after a finite time, all the 
messages are in buffers that are suitable for them. 

Proof When the routing table are stabilized some of the messages still are 
in buffers that are not suitable for then. However, since the routing tables are 
stabilized, every message is generated in a suitable buffer, hence the number of 
messages that are in buffers that are not suitable for them does not increase. 
In another hand. According to Lemma [5l every message that is in the wrong 
direction will undergo exactly one route change and hence all the wrong messages 
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that are in the system wih be in the right direction in a finite time and the lemma 
holds. 

□ 

From there, it is important to show that any processor can generate a message 
in a finite time. From Lemma [6l all the messages are in suitable buffers in a finite 
time. Since the PIF waves are used for route changes only, then no PIF wave 
will be initiated. That what we show in the two following lemmas: 

Lemma 7 In the case where PIF- Request =true, it will he set at false in a finite 
time. 

Proof. Note that in the case where PIF-request is true and the B- Action 
of the initiator is enabled on po, PIF- Request will be set at false when the 
B- Action is executed (see B- Action of the initiator). Otherwise two cases are 
possible according to the state of the initiator: 

— = (5,?). In this case PIF- Request is set at false by the forwarding 
algorithm by executing i?13 (i?13 is enabled on po and the daemon is weakly 
fair). 

— = {C^ NULL). If INp^ contains a messages and the destination of he 
message is not po then PIF- Request will be set at false by the PIF algorithm 
once the PIF-wave is initiated. However in the case where the input buffer 
of po is empty or contains a message to po then RIA is enabled and since the 
daemon is weakly fair i?14 will be executed on po and hence PIF-Request is 
set at false. 

From the cases above we can deduct that if PIF-Request is true then it will be 
set at false in a finite time and the lemma holds. □ 



Lemma 8 When the routing tables are stabilized and all the messages are in 
suitable buffer, no PIF wave is initiated. 

Proof. According to Lemma [71 PIF-Request will be set at false in a finite 
time. Note that the only rule that set PIF-Request at true is R7. However R7 is 
never enabled since all the messages on the chain are in suitable buffer and since 
the routing tables are correct (all messages are generated in suitable buffer). 
Thus the lemma holds. □ 

From this point, the fair pointer mechanism cannot be disrupted by the PIF 
waves anymore. So, the fairness of message generation guarantees the following 
lemma: 

Lemma 9 Any message can be generated in a finite time under a weakly fair 
daemon. 

Proof. According to Lemma [H when the routing tables are stabilized and 
when all the messages are containing in buffers that are suitable for them no PIF 
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and no Road-change are executed. In another hand since the routing tables are 
stabihzed and since the buffer graph of the chain consists on two disjoint chains 
(it is a DAG) then no deadlock happens and all the messages progress in the 
system. Now suppose that the processor p wants to generate a message. Recall 
that the generation of a message m for the destination d is always done in the 
output buffer of the processor p connected to the link (p, q) such as Nextp{d) = q. 
Two cases are possible 

1. OUTp{q) = e. In this case, the processor executes either Rl or R3 in a finite 
time, the result of this execution depends on the value of the pointer. Two 
cases are possible: 

— the pointer refers to Rl. Then p executes Rl in a finite time and we 
obtain the result. 

— the pointer refers to R3. Then p executes R3 in a finite time. Hence 
OUTp{q) / e and we retrieve case [21 Note that the fairness of the 
pointer guarantees us that this case cannot appear infinitely. 

2. OUTp{q) ^ e. Since all the messages move gradually in the buffer graph we 
are sure that OUTp{q) will be free in a finite time and we retrieve case [H 

We can deduct that every processor can generate a message in a finite time. □ 
Due to the color management (Function Choice{c))^ the next lemma follows: 

Lemma 10 The forwarding protocol never duplicates a valid message even if A 
runs simultaneously. 

Proof. Three cases are possible: 

— m is in INp{q). According to the rules that are enabled on three cases are 
possible 

• the message is consumed {R2 is executed ) hence the message m is deleted 
from INp{q) since a new value overwrites since INp{q) = OUTq{p) (Note 
that this happen only when OUTq{p) ^ INp{q)). 

• RS is executed . The message is copied in EXTp (for the processor 
Po) and deleted from INp{q) since a new value overwrites {INp{q) = 
OUTq{p)) in a sequential manner. 

• R4: is executed. The message is put in this case in OUTp{q') and deleted 
from INp{q) in a sequential manner hence only one copy is kept {q^ G 
Np/{q}). Note that these two rules are not enabled only if OUTq{p) does 
not contain the same message. 

— m is in OUTp{q). In this case the message m is copied in the input buffer of 
the processor q {INq{p)). Hence two copies are in the system. However the 
message in INq{p) is not consumed and not transmitted unless the copy in 
OUTp{q) is deleted (see the rules R2 and RA). 
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— m is in EXTp. In this case the message is either deleted or put in OUTp{q). 
Since this operation is a local operation (the copy is done between two buffer 
of the same processor) then the message is copied in the new buffer and 
deleted from the previous one in a sequential manner. 

From the cases above we can deduct that no message is duplicated in the 
system. Hence m is delivered at most once to its destination. □ 

From Lemma [9l any message can be generated in a finite time. From the 
PIF mechanism and its synchronization with the forwarding protocol the only 
message that can be deleted is the message that was in the extra buffer at the 
initial configuration. Thus: 

Lemma 11 Every valid message (that is generated by a processor) is never 
deleted unless it is delivered to its destination even if Rtables runs simulta- 
neously. 

Proof. The proof is by contradiction, suppose that there is a message m 
that is deleted without being delivered to its destination. 

By construction of i?3, this cannot be a result of an internal forwarding since 
the message m is first of all copied in the Output-buffer OUTp{q) and then erased 
from the Input-buffer INp{q') since a new value overwrites it. Note that these 
two rules are enabled only if OUTp{q)=I Nq{p) or OUTp{q) = e. Hence when the 
message m is copied in the OUTp{q) no message is deleted (one copy remains in 
INq{p) in the case where INq{p) = OUTp{q)). 

By the construction of Rule i?4, the message is only copied in the Input-Buffer 
and not deleted from the Output-buffer at the neighbour processor simultane- 
ously (the only rules that delete a message from the Output-buffer are R5 and 
R3 and the guards of these rules are not verified when RA is enabled). 

If Rb is enabled in processes that means that OUTq{p) = INp{q) and 
INp{q') = e^q' ^ Np/ {q]. When one of these two rules are enabled, OUTp{q) = e. 
However according to the color management (Function Choice (c)), we are sure 
that a copy of the message that was in OUTp{q) is in INq{p). 

By the construction of the rules R6 and i?ll, this cannot be the result of 
the execution of these two rules because the message that is in INp{q) such as 
Pq and p is not the destination, is copied in the Output buffer and deleted from 
the Input buffer sequentially and then p copies the message that is in OUTq{p) 
in INp{q) , so no message is deleted. 

Concerning i?12, EXTp such as po contains the message m and Sp = (C, ?), 
which means that no PIF is executed. However, for po, a message in INp{q) is 
copied in EXTp (in the case where p is not the destination) only if RS is enabled, 
however, when R8 is enabled B-intiator is enabled as well. Since in this case 
the two rules are executed at the same time, hence Sp = (5,?). Now, for the 
processor p. Since the PIF is a valid PIF, when the C- Action of p is enabled 
at the same time as the rules i?10 or i?9. If i?10 is executed then EXTp — e 
and Sp = (C.NULL) (since OUTp{q) = e or OUTp{q) = INq{p)), which is a 
contradiction, since in our case EXTp ^ e. If is executed, then the message 
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in the extra buffer of p {EXTp) is copied in OUTp{q)^ EXTp becomes free and 
Sp = (C, ?), which is a contradiction with our case. Hence we are sure that the 
message that is in the extra buffer of p is a message that was not generated 
by a processor. Hence when R12 is executed, this message is deleted (no vahd 
message is deleted). 

By the construction of the two rules R8 and i?9, No valid message is deleted 
by the execution of the two rules, since the message is copied in the extra buffer 
{RS) or in the Output buffer {R9) and then it is deleted from the Input buffer 
Ir8) or the extra buffer {R9). 

Concerning the rule i?10, according to LemmalU when the message that is in 
EXTp is valid, when the C- Action of the initiator is enabled either OUTp{q) = e 
or OUTp{q) = INq{p). However no such buffers exist. Hence the message in the 
extra buffer of p is not a valid message (it is not generated by a processor). Hence 
it can be deleted. 

We can deduct from all the cases above that no message that is generated 
by a processor is deleted, hence the lemma holds. 

□ 



Theorem 1. The proposed algorithm (Algorithms^ and\^ is a snap- stabilizing 
message forwarding algorithm (satisfying SP) under a weakly fair daemon. 

Proof From Lemma [9l any message can be generated in a finite time. 
From Lemma [Til every valid message is never deleted unless it is delivered to 
its destination even if Rtables runs simultaneously. From Lemma [TOl no valid 
message is duplicated. Hence, the theorem holds. □ 

Remarque 

For any processor p. Forwarding protocol delivers at most 4n— 3 invalid messages. 

Proof Assume that in the initial configuration all the buffers contain a 
message, since these messages were not generated by the processors of the system, 
they are invalid messages. Suppose that the destination of the message m in 
INp{q) is the processor p such as = and Sq = (5,-1). Suppose that the 
daemon activates p which executes R2 and the F- action (it is a leaf) . Hence the 
message m is consumed and INp{q) = OUTq{p). Hence OUTq{p) becomes free 
and the C-action of the initiator is enabled, q will copy then the message from 
EXTq in OUTq{p) and will execute the C- Action. In another hand, since there 
is no way to know if the messages are valid or not, they all be treated as if they 
are valid. Since the forwarding algorithm is snap stabilizing, all the messages 
that were in the buffer of the chain at the beginning are delivered. Since there is 
4n-3 buffers in the system, then 4n-3 invalid messages can be delivered and the 
lemma holds. 
□ 
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5 Dynamicity 

In dynamic environments, processors may leave or join the network at any time. 
To keep our solution snap stabilizing we assume that there are no crashes and if a 
processor wants to leave the network (disconnect), it releases its buffers (it sends 
all the messages it has to send and wait for their reception by its neighbours) 
and accepts no more message before leaving. 

In this discussion we assume that the rebuilt network is still a chain. It is 
fundamental to see that in dynamic systems the problem of keeping messages for 
ghost destinations with the hope they will join the network again and the lack of 
congestion are contradictory. If there is no bound on the number of leavings and 
joins this problem do not admit any solution. The only way is to redefine the 
problem in the context of dynamicity. For example we can modify the second 
point of the specification (SP) as follows: A valid message m generated by the 
processor p to the destination q is delivered to in a finite time if m, p and 
q are continuously in the same connected component during the forwarding of 
the message m. Even if that could appear very strong, this kind of hypothesis is 
often implied in practice. However we can remark that this new specification is 
equivalent to SP in static environments. Our algorithm can easily be adapted 
in order to be snap stabilizing for this new specification in dynamic chains. 

Thus we can now delete some messages as follows: we suppose that every 
message has an additional boolean field initially set to false. When a message 
reaches an extremity which is not its destination we have two cases: (i) The 
value of the boolean is false, then the processor sets it to true and sends it in 
the opposite direction, (ii) The value of the boolean is true, then the processor 
deletes it (in this case, if the message is valid, it crossed all the processors of the 
chain without meeting its destination). 

Finally, in order to avoid starvation of some processors, the speed of joins 
and leavings of the processors has to be slow enough to avoid a sequence of PIF 
waves that could prevent some processors to generate some messages. 

6 Conclusion 

In this paper, we presented the first snap-stabilizing message forwarding protocol 
that uses a number of buffers per node being independent of any global param- 
eter. Our protocol works on a linear chain and uses only 4 buffers per link. It 
tolerates topology changes (provided that the topology remains a linear chain). 
This is a preliminary version to get the same result on more general topologies. 
In particular, by combining a snap-stabilizing message forwarding protocol with 
any self-stabilizing overlay protocols {e.g., [13] for DHT or |14|15|16] for tries) ^ 
we would get a solution ensuring users to get right answers by querying the 
overlay architecture. 
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